What Quality Control Mechanisms do We Need for High-Quality Crowd Work?

Crowd sourcing and human computation has slowly become a mainstay for many application areas that seek to leverage the crowd in the development of high quality datasets, annotations, and problem solving beyond the reach of current AI solutions. One of the major challenges to the domain is ensuring high-quality and diligent work. In response, the literature has seen a large number of quality control mechanisms each voicing (sometimes domain-specific) benefits and advantages when deployed in largescale human computation projects. This creates a complex design space for practitioners: it is not always clear which mechanism(s) to use for maximal quality control. In this article, we argue that this decision is perhaps overinflated and that provided there is "some kind" of quality control that this obviously known to crowd workers this is sufficient for "high-quality" solutions. To evidence this, and provide a basis for discussion, we undertake two experiments where we explore the relationship between task design, task complexity, quality control and solution quality. We do this with tasks from natural language processing, and image recognition of varying complexity. We illustrate that minimal quality control is enough to repel constantly underperforming contributors and that this is constant across tasks of varying complexity and formats. Our key takeaway: quality control is necessary, but seemingly not how it is implemented.

The experiences of adults with learning disabilities in the UK during the COVID-19 pandemic: qualitative results from Wave 1 of the Coronavirus and people with learning disabilities study

Purpose The purpose of this paper is to present data about the experiences of adults with learning disabilities during the COVID-19 pandemic across the UK. Design/methodology/approach Interviews were conducted with 609 adults with learning disabilities. Family carers and support staff of another 351 adults with learning disabilities completed a proxy online survey. The data were collected between December 2020 and February 2021 and concerned both worries/negatives and anything positive that had happened because of the COVID-19 pandemic. Findings Social isolation was the most commonly reported worry/negative for adults with learning disabilities, with other frequently reported worries/negatives including: changes to/loss of routine;loss of support/services;and decreased health/well-being/fitness. A large proportion of participants indicated that nothing positive had happened because of COVID-19, but some positives were reported, including: digital inclusion;more time spent with important people;improved health/well-being/fitness;and, a slower pace of life. Practical implications Future pandemic planning must ensure that adults with learning disabilities are supported to maintain social contact with the people who matter to them and to support their health and well-being (including maintaining access to essential services and activities). Some adults with learning disabilities may benefit from additional support to improve their digital confidence and access. This may in turn enable them to maintain contact with family, friends and support services/activities. Originality/value This is the largest study about the experiences of adults with learning disabilities during the COVID-19 pandemic in the UK. The authors primarily collected data directly from adults with learning disabilities and worked with partner organisations of people with learning disabilities throughout the study.

The willingness of UK adults with intellectual disabilities to take COVID-19 vaccines

Abstract Background Given the much greater COVID-19 mortality risk experienced by people with intellectual disabilities (ID), understanding the willingness of people with ID to take a COVID-19 vaccine is a major public health issue. Method In December 2020 to February 2021, across the United Kingdom, 621 adults with ID were interviewed remotely and 348 family carers or support workers of adults with ID with greater needs completed an online survey, including a question on willingness to take a COVID-19 vaccine if offered. Results Eighty-seven per cent of interviewees with ID were willing to take a COVID-19 vaccine, with willingness associated with white ethnicity, having already had a flu vaccine, gaining information about COVID-19 from television but not from social media, and knowing COVID-19 social restrictions rules. A percentage of 81.7% of surveyed carers of adults with ID with greater needs reported that the person would be willing to take a COVID-19 vaccine, with willingness associated with white ethnicity, having a health condition of concern in the context of COVID-19, having had a flu vaccine, being close to someone who had died due to COVID-19, and having shielded at some point during the pandemic. Conclusions Reported willingness to take the COVID-19 vaccine is high among adults with ID in the United Kingdom, with factors associated with willingness having clear implications for public health policy and practice.